> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/Anny26022/chartsmaze_clone/llms.txt
> Use this file to discover all available pages before exploring further.

# enrich_fno_data.py

> F&O Data Enrichment - Adds F&O flags, lot sizes, and next expiry dates to the master stock database

## Overview

`enrich_fno_data.py` is a Phase 4c pipeline script that enriches the master stock database with Futures & Options (F\&O) data. It fetches current lot sizes and expiry calendar information from Dhan's API and maps them to stocks flagged as F\&O-eligible in the master ISIN map.

<Info>
  **Pipeline Position:** Phase 4c - Final enrichment step before data export

  **Critical Function:** Adds trading-essential F\&O metadata required for options and futures analysis
</Info>

## Purpose

This script:

* Identifies F\&O-eligible stocks from `master_isin_map.json`
* Fetches real-time lot sizes from Dhan
* Retrieves next expiry dates from the F\&O calendar
* Enriches `all_stocks_fundamental_analysis.json` with F\&O metadata

## Input Files

<ParamField path="all_stocks_fundamental_analysis.json" type="JSON" required>
  Master stock database containing all fundamental and technical data
</ParamField>

<ParamField path="master_isin_map.json" type="JSON" required>
  ISIN mapping file with `FnoFlag` field (1 = F\&O eligible, 0 = not eligible)
</ParamField>

## Output Files

<ResponseField name="all_stocks_fundamental_analysis.json" type="JSON">
  Updated master file with three new fields added to each stock record:

  * `F&O`: "Yes" or "No"
  * `Lot Size`: Integer lot size or "N/A"
  * `Next Expiry`: Date string (YYYY-MM-DD) or "N/A"
</ResponseField>

## Processing Logic

### 1. Build ID Extraction

Dynamically fetches the Next.js `buildId` from Dhan's lot size page:

```python theme={null}
def get_build_id():
    """Dynamically fetch the Next.js buildId from a Dhan page."""
    url = "https://dhan.co/nse-fno-lot-size/"
    try:
        response = requests.get(url, headers={"User-Agent": get_headers()["User-Agent"]}, timeout=10)
        match = re.search(r'"buildId":"([^"]+)"', response.text)
        return match.group(1) if match else None
    except:
        return None
```

### 2. Lot Size Fetching

Retrieves current month lot sizes for all F\&O instruments:

```python theme={null}
def fetch_lot_sizes(build_id):
    """Fetch current F&O lot sizes. Returns {symbol: lot_size_current_month}."""
    lot_map = {}
    if not build_id:
        return lot_map

    url = f"https://dhan.co/_next/data/{build_id}/nse-fno-lot-size.json"
    headers = {"User-Agent": get_headers()["User-Agent"]}

    try:
        r = requests.get(url, headers=headers, timeout=15)
        if r.status_code == 200:
            data = r.json()
            instruments = data.get("pageProps", {}).get("listData", [])
            for item in instruments:
                sym = item.get("sym")
                fo_contracts = item.get("fo_dt", [])
                if sym and fo_contracts:
                    # First contract = current month lot size
                    lot_map[sym] = fo_contracts[0].get("ls")
    except:
        pass

    return lot_map
```

### 3. Expiry Date Fetching

Retrieves the next upcoming expiry date for each F\&O symbol:

```python theme={null}
def fetch_next_expiry(build_id):
    """Fetch F&O expiry calendar. Returns {symbol: next_expiry_date}."""
    expiry_map = {}
    if not build_id:
        return expiry_map

    url = f"https://dhan.co/_next/data/{build_id}/fno-expiry-calendar.json"
    headers = {"User-Agent": get_headers()["User-Agent"]}

    try:
        r = requests.get(url, headers=headers, timeout=15)
        if r.status_code == 200:
            data = r.json()
            expiry_raw = data.get("pageProps", {}).get("expiryData", {}).get("data", [])

            from datetime import datetime
            today = datetime.now().strftime("%Y-%m-%d")

            for exchange_data in expiry_raw:
                for exp_group in exchange_data.get("exps", []):
                    for item in exp_group.get("explst", []):
                        sym = item.get("symbolName")
                        exp_date = item.get("expdate")
                        if sym and exp_date and exp_date >= today:
                            # Keep the nearest future expiry
                            if sym not in expiry_map or exp_date < expiry_map[sym]:
                                expiry_map[sym] = exp_date
    except:
        pass

    return expiry_map
```

### 4. Data Enrichment

Maps F\&O data to stocks based on the FnoFlag:

```python theme={null}
# Enrich master JSON
enriched = 0
for stock in master_data:
    sym = stock.get("Symbol")

    if sym in fno_symbols:
        stock["F&O"] = "Yes"
        stock["Lot Size"] = lot_map.get(sym, "N/A")
        stock["Next Expiry"] = expiry_map.get(sym, "N/A")
        enriched += 1
    else:
        stock["F&O"] = "No"
        stock["Lot Size"] = "N/A"
        stock["Next Expiry"] = "N/A"
```

## Fields Added

<ResponseField name="F&O" type="string">
  Indicates F\&O eligibility

  * `"Yes"` - Stock is F\&O eligible
  * `"No"` - Stock is not F\&O eligible
</ResponseField>

<ResponseField name="Lot Size" type="string | number">
  Current month lot size for F\&O contracts

  * Integer value (e.g., `500`, `1000`) for F\&O stocks
  * `"N/A"` for non-F\&O stocks
</ResponseField>

<ResponseField name="Next Expiry" type="string">
  Next upcoming expiry date

  * Date in YYYY-MM-DD format (e.g., `"2026-03-26"`) for F\&O stocks
  * `"N/A"` for non-F\&O stocks
</ResponseField>

## Usage Example

```bash theme={null}
python enrich_fno_data.py
```

**Expected Output:**

```
Found 243 F&O eligible stocks from ISIN map.
Fetching Dhan buildId...
  BuildId: abc123xyz
Fetching F&O lot sizes...
  Got lot sizes for 243 instruments.
Fetching F&O expiry calendar...
  Got expiry dates for 243 instruments.
Successfully enriched 243 F&O stocks in master JSON.
```

## Error Handling

* **Missing master\_isin\_map.json**: Script continues but no stocks are marked as F\&O eligible
* **Build ID fetch failure**: Returns empty lot size and expiry maps
* **API timeout**: Individual fetch functions fail gracefully, returning empty dictionaries
* **Symbol mismatch**: Stocks not found in Dhan data receive "N/A" values

## Data Source

* **F\&O Eligibility**: `master_isin_map.json` (FnoFlag field)
* **Lot Sizes**: Dhan NSE F\&O Lot Size API
* **Expiry Dates**: Dhan F\&O Expiry Calendar API

## Related Scripts

* [bulk\_market\_analyzer.py](/api/bulk-market-analyzer) - Creates the master stock database
* [process\_market\_breadth.py](/api/process-market-breadth) - Uses F\&O flags for analytics